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ABSTRACT 



In order to provide teachers and students with electronic 
learning environments that support mentoring and collaboration through 
electronic means, the authors developed software that supports same 
time/different place educational collaborative activities over the Internet. 
These activities focus on teaching students how to organize and systematize 
their explorations for information and to share this information with members 
of a group. The software is composed of an interface that connects the 
hyperlink, text, graphics, video, and sound capabilities of a browser to a 
World Wide Web client-server environment and is augmented by local electronic 
help tools and computer supported collaborative tools. Collaborative 
activities are supported through specially designed whiteboard, chat, file, 
and application sharing tools. Students can access online schedules and 
determine whether faculty or other students are available. The system is also 
equipped with management and record keeping capabilities that allow teachers 
or trainers to analyze various collaborative activities. Students enrolled in 
a variety of courses at the University of North Texas used the system to 
collaborate with other students and teachers. Information was logged 
concerning overall student performance, activities performed while using the 
collaborative systems, and attitudes related to technology and collaboration. 
Results were analyzed to determine which factors contributed to 
success ful/unsuccessful collaboration. (MES) 
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TO THE EDUCATIONAL RESOURCES 

INFORMATION CENTER (ERIC) AU . . . 

Abstract: While the growth of the Internet and collaborative software has greatly 

impacted the way students work together, little is known about what actually works and 

what doesn't. Some researchers argue that both structural and cultural problems impact 

effectiveness, while others suggest that a lack of "critical mass" may be affecting the 

results. This paper suggests that all of these things are true. In also contends that the 

type of task required of the group and collaborative style influences effective real-time 

collaboration. While this paper confirms that collaborative interfaces help most people 

work more effectively, it also shows that some groups are less successful, have fewer 

interactions, and remain hostile towards the system even though they complete the 

required task. More importantly, the results of our research raise additional questions 

about how technology can support distributed groups, and what is the nature of the 

mismatches. 



° Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



Introduction 

As the number of distance learning and online courses continues to increase, we are being 
forced to re-examine the way we teach students to think. The idea of the “classroom teacher” is 
gradually being replaced by a web page or “talking head” that emanates from a remote site. 
Because of the rapid growth of this type of technology, we are beginning to question whether 
technology can support the in-depth contact between subject-matter expert and a small group of 
learners that is at the heart of our nation's peerless reputation in higher education. While both the 
Internet and other types of distance learning systems provide interesting pedagogical experiences, 
they lack many of the features that are associated with traditional educational classrooms such as 
laboratory exercises and small group discussions. If one of our goals is to improve education 
(Willis 1993), then we need find ways to provide teachers and students with electronic learning 
environments that support both mentoring and collaboration through electronic means. 
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In order to address this challenge, the authors developed special software that supports same 
time/different place educational collaborative activities over the Internet. These activities focus 
on teaching students how to organize and systematize their explorations for information and to 
share this information with members of a group. The software is composed of a unique interface 
that connects to the hyperlink, text, graphics, video and sound capabilities of a browser to a 
World Wide Web (WWW) client-server environment, and is augmented by local electronic help 
tools and computer supported collaborative tools. Collaborative activities are supported through 
specially designed whiteboard, chat, file and application sharing tools. Students can access online 
schedules and determine whether faculty or other students are available. The system is also 
equipped with both management and record keeping capabilities that allow teachers or trainers to 
analyze the various collaborative activities. Over 200 students enrolled in a variety of different 
courses at the University of North Texas used the system during the past year to collaborate with 
both students and teachers throughout the campus and different parts of the Metroplex. 
Information was logged concerning overall student performance, activities performed while using 
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the collaborative system, and attitudes related to technology and collaboration. Results from 
these history files were then analyzed to determine which factors contributed to 
successful/unsuccessful collaboration. 

While the past year’s research show that collaborative interfaces help most groups work 
effectively, it also shows that some groups are less successful, perform fewer group interactions 
as compared to successful groups, or are uncomfortable with the system even after several hours 
of training. For example, we observed that some individuals completed the stated task but never 
shared information with other members of the group. Other subjects were disruptive, eventually 
causing the group to fail. In contrast, there were groups who learned how to complete tasks 
successfully, even when faced with adverse network conditions. Although the results of this 
research show that most distributed groups produced work that was indistinguishable from groups 
doing the same work face-to-face, some groups’ performed poorly, some had members who never 
participated, and some work teams complained about the system, even though they performed 
well. Thus, over the course of two years, the authors have discovered many factors that both 
encourage and deter collaboration among students. The nature of these factors and how they 
impact effectiveness are discussed below. 



Relevant Literature 

Much has already been accomplished in demonstrating the potential of using the Internet for 
educating, sharing, connecting, and communicating. For example, students enrolled in political 
science courses use the net to access large databases (Baker 1993; Newton 1993), students in high 
school use it to contact experts in the field (Press 1995), and students in business use it to learn 
about changing stock prices (Winner 1994). An ever increasing number of virtual classrooms are 
now available to students living in different cities or states (Huang 1997; Foster 1995). 
However, increasing educational opportunities have resulted in decreasing classroom time, and 
hence, decreasing the number of occasions where students and teachers interact with one another 
face-to-face. Thus, the question remains of how to make distance leaning environments support 
real-time educational activities such as academic advising, mentoring, laboratory experiments and 
small group exchanges (Reinhart 1995). While we know that network technology can be used to 
empower learners with the latest tools of the trade, we also know that it lacks the personal 
element that can give meaning to an educational experience. Thus, our prototypes and models 
must clearly demonstrate how network technology can evolve into an educational ecosystem of 
information, computation, communication, collaboration, and innovative mentoring. At the same 
time, the models must emphasize information access and visualization to provide easy derivation 
of an interactive information experience. 

In response to these problems, the authors engaged in a number of innovative collaborative 
software projects sponsored by various funding agencies over the past several years (Swigger & 
Brazile 1995; Swigger et al., 1997). In order to enhance students' understanding of the 
cooperative process, we built several computer-supported cooperative interfaces that teach 
students how to cooperate more effectively in the accomplishment of specific tasks (e.g., library 
database searching, requirements elicitation, and cost center analysis). The cooperative interfaces 
all emphasize shared window systems (i.e., What You See Is What I See) that allow students who 
are geographically distanced from one another to work together at the same time. While several 
of the original interfaces proved effective, they did not support shared activities over the Internet. 
Thus, we developed an Internet-based tool that supports real-time collaborative learning over the 
network. The major focus of the research is on showing students and teachers how to work 
together over the Internet, and how to locate, synthesize, and analyze vast amounts of available 
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information. The long-term goal of the project, therefore, is to study what factors (in terms of 
both the interface and instructional materials) lead to effective collaborations. Thus, we are 
analyzing how teachers and students move from their specialized classrooms into shared, multi- 
user settings where diverse groups can collectively solve problems. 



Collaborative Software 

As previously mentioned, the project’s major objectives are (1) to develop software that 
allows students to collaborate, and (2) to determine whether the software can be effective in an 
educational setting. Thus, the first task, to develop software, has resulted in a series of programs 
that allows students to access special tools, libraries, and the network, to pose questions and 
conduct exchanges with teachers and other students in real-time. As a consequence, the system 
supports students who are located in different geographical areas, but who need real-time (i.e., 
What You See Is What I See) access to information from a number of different sources, including 
a university, the Web, and local networks as well as other students and teachers. Students are free 
to perform activities, find information, conduct experiments, and engage in collaborative 
activities to obtain information. Since the system supports both synchronous and asynchronous 
communication, the collaborative activities can occur either at the same or different times. 

The current application comes equipped with a number of collaborative capabilities that 
enable teachers and students located in different geographical locations to “enter” each other’s 
worlds. For example, the system allows groups to share applications and perform a specific task 
such as completing an experiment, writing a paper, using a software package. Thus, the major 
software components include (1) a special graphical user interface, (2) support programs that 
manage the communications and data gathering functions, and (3) administrative programs that 
support activities such as student/group registration and report generation. The special graphical 
user interface consists of a shared Browser for the WWW, and a set of collaborative tools that 
permit groups to chat, use a whiteboard, and share both applications and files. The support 
programs that manage the communications and data gathering functions include a polling 
application that maintains the user and group lists, and a suite of programs that create, maintain, 
and update the database. Finally, the system performs administrative functions such as 
installation, student and group registration, and report writing. 



Evaluation 

As previously mentioned, a major component of this research is the evaluation of the 
effectiveness of the collaborative interface to determine what is and is not effective. During the 
past two years, over 400 students have used the collaborative software in seven different courses. 
Courses using the collaborative courses were compared to control courses using a pre-test post- 
test, matched controls design. The evaluation was designed to determine the effect of the 
collaborative technology on student learning outcomes, student attitudes, and attitudes about 
collaboration. Results from comparisons of just the performance variables (i.e., that course 
grades and scores on the questions concerning analysis and synthesis of information) indicate no 
significant difference between the control and experimental groups. 

We also collected information from the computer sessions through our data collection 
programs. The system logs information such as source, time, and type of interaction. Thus, we 
were able to capture a detailed record of the types of activities that students (and groups) perform 
while using the interface. Analysis of this data indicates that group chat was the most frequently 



used collaborative tool regardless of course content or activity. The shared web browser was the 
second most frequently used tool, while usage totals for all other tools corresponded to the type of 
task that was performed by a particular group. The system log information was also correlated 
with factors such as age, a student’s major, and gender and indicated that these variables often 
impact a group’s productivity (as measured by total time to complete a task and the activities 
performed during the task). For example, students enrolled in non-technical courses used the 
chat tool more frequently and longer than students enrolled in science course. Groups who were 
dissatisfied with the system did not necessarily perform less well than groups who were satisfied. 
And finally, as network delays increased, remote groups spent more time managing their work or 
verifying that a message had been either sent or received. Although our observations seem sound, 
they are based on a limited amount of data. In the next year, we will gather additional 
information about a group’s work, learning, and communication style to determine their effect on 
distributed problem solving. 

From both observation and data gathered during the project, a number of conclusions were 
made concerning how students used the interface and which parts were successful and 
unsuccessful. These conclusions are as follows: 

1. Students seem unable to manage multi-tasking or multiple windows. As new applications 
are opened, existing windows or applications are hidden. Once removed, students mistakenly 
believe that the application has shut down and must be re-opened. This can reek havoc in a 
synchronous communication environment. 

2. Students seem unable to distinguish between the different modes of a computer. For 
example, they fail to understand that there is a difference between “shared” and “individual” 
modes of work. 

3. Students need to know how to map the computer-collaborative tools onto the specific tasks. 
(I.e., Which collaborative tool should be used for which task?). 

4. Students become frustrated when they do not know what other members of the group are 
doing. As the students become more proficient with the software, they learn to use awareness 
tools that provide them with information about the other members of the group. 

5. Students lack metaphors for shared software applications. As a result, they have no 
established protocol for how to interact with team members while using this type of software. 

We also examined the various collaborative exercises used in this project to determine which 
factors contributed to success. Examples of these activities can be found on the project’s web 
page (i.e., www.vcuproject.edu). The types of activities developed by different faculty tend to 
vary according to the nature of a particular discipline and the faculty member’s particular style. 
However, based on both observation and data, we are able to make a number of conclusions about 
what types of tasks were most successful. For example, successful collaborative experiences 
tended to be both extremely focused and small. Initially, faculty members modeled their 
collaborative exercises after classroom exercises in which both groups and teacher are free to 
roam around the room. Unlike the classroom, however, a teacher cannot always sense when a 
group is frustrated or confused and must often wait until it is too late to correct the problem. As a 
result, many of the computer-supported collaborative activities were scaled down to 
accommodate the loss of visual cues. At the same time, the set of instructions provided to 
students were greatly increased and posted on both the faculty and course web pages. 
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We also found that it was not always possible to adapt a classroom group activity to a 
computer-supported collaborative environment. Tasks that worked well in the classroom did not 
always translate into the new environment. One such activity was an environmental science 
project that asked students to collect data on a number of different events and then share this data 
with other members of the group. Given the location of the data collection source (e.g., an 
outdoor pond), it was neither convenient nor possible for students to share their data in a 
machine-readable format. As a result, the collaborative software became more a hindrance than 
help in supporting collaboration. 

Data on faculty satisfaction also tended to vary. Faculty were surprised that Internet 

communication was sometimes slow and inconsistent. While some faculty experienced no 
problems with the hardware or software, others were overwhelmed with the technology. 
Successful experiences with both software and student collaboration tended to occur when the 
faculty member was well organized, and the student activities were well structured, small, and 
self-contained. As the complexity of the educational activity increased, the level of unsatisfaction 
among both faculty and students also increased. Finally, our interviews with faculty that 
participated in this project clearly indicate that most do not realize that collaborative skills have 
not been mastered at an earlier age. However, our data suggests that most students lack both 
organizational and interaction skills and, as a result, often end up working alone to complete the 
task. For example, the most frequent statement recorded in or study’s database was the question: 
“Ok, what do we do now?” This is then followed by a series of responses, all echoing the phrase, 
“I don’t know.” As a result, the training sessions included information about the use of the 
system and how to collaborate. For example, students were shown how to use the chat tool to 
establish their group’s procedures and distribute the work. After such training, the collaborative 
exercises were much more successful. 



Conclusions and Impact of Research 

In conclusion, we believe that we have identified a number of factors that contribute to 
successful computer-supported collaboration. After developing a system that supports real-time 
collaboration for students in different locations, we examined how students enrolled in a number 
of different courses at the University of North Texas used the software to complete specific tasks. 
While our research shows that the collaborative system helps most groups work effectively, it 
also shows that some groups are less successful, perform at a slower rate than face-to-face 
groups, or are intimidated by the system even after several hours of training. Success with the 
system seems to be affected by task, training, and experience. Knowledge of how to perform as a 
leader also seems to be an important factor. 

Furthermore, groups’ preferences for and use of certain collaborative tools, as well as their use 
of features within those tools, greatly varies. However, judged by the total time spent completing 
the task and by their scores on a series of related questions, performance seems unaffected by 
whether distributed groups used all or some of the collaborative tools. Our studies also show that 
the collaborative interface allowed groups to brainstorm and co-author documents and, in doing 
so, encourages both talking and doing in the same meeting. On the other hand, because of the 
often-restrictive nature of the interface, groups are forced to talk to each other via their fingers 
rather than their mouths. As a consequence, team members with inadequate typing skills feel 
inhibited or restrained. In addition, we noticed that many students learned to adjust their 
communication style and adapt to the more delayed mode required by the chat tool. 
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In our research, we have found that while software allows groups to work more effectively, 
group performance is often impacted by a variety of individual personality, communication, and 
work style preferences. We have also observed that groups tend to react to individual preferences 
in a number of different ways. Some groups engage in additional procedural or management 
activities, while others perform fewer clarification or solution activities. It is our observation that 
the need for low cost, distributed education occurs frequently in this technological society. Yet 
rarely do we use network technology to actually meet these educational goals, nor do we provide 
students and teachers with the tools to communicate in this mode. As such, this research should 
provide insight and software tools for broad-based interactive learning by electronic means. 
Moreover, it provides a rich research environment that allows us to investigate various issues 
related to improving the performance of student and organizational teams that work together 
across distances. 
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